Graph Attention for Automated Audio Captioning

State-of-the-art audio captioning methods typically use the encoder-decoder structure with pretrained neural networks (PANNs) as encoders for feature extraction. However, convolution operation used in PANNs is limited capturing long-time dependencies within an signal, thereby leading to potential performance degradation captioning. This letter presents a novel method using graph attention (GraphAC) based In encoder, module introduced after learn contextual association (i.e. dependency among features over different time frames) through adjacency graph, and top- k mask mitigate interference from noisy nodes. The learnt leads more effective representation node aggregation. As result, decoder can predict important semantic information about acoustic scene events on associations learned signal. Experimental results show that GraphAC outperforms state-of-the-art encoders, thanks incorporation of into encoder source code available at https://github.com/LittleFlyingSheep/GraphAC.